Morphological Analysis and Diacritical Arabic Text Compression

Morphological analysis of Arabic words allows decreasing the storage requirements of the Arabic dictionaries, more efficient encoding of diacritical Arabic text, faster spelling and efficient Optical character recognition. All these factors allow efficient storage and archival of multilingual digital libraries that include Arabic texts. This paper presents a lossless compression algorithm based on the affix analysis that takes advantage of the statistical studies of the diacritical Arabic morphological features. The algorithm decomposes a given Arabic word into its root and its affixes. The affixes (prefix, infix, and suffix) are the redundant elements of the word. The roots are stored in the root dictionary. Also, we maintain categorized affix dictionaries and their valid combinations to validate and generate the morphological forms during encoding and decoding using a list of patterns. Since our goal is lossless reproducible Arabic text, stemming is not an option and noise words (high frequency words) cannot be filtered out. The size of the obtained root dictionary is about 8000 three-character roots and 700 four character roots. We also code the most frequently occurring diacritical bigrams (biliterals) and trigrams (triliterals) with unused codewords in ASCII, ASMO-449, and Unicode standard codes. Using combined methods of root dictionaries and the proposed coding scheme, compression ratios of proper Arabic text compare favorably with other unigram non-diacritical methods. Keyword: Compression, Affixes, Morphological Analysis, Dictionary, Root, Diacritics, lexicon, spelling, archival, digital library.